Image processing apparatus, image processing method, and program for displaying a preview of a document region and a recognized content processing

ABSTRACT

An image processing apparatus prints a processing instruction sheet for instructing to perform processing on a document to be processed. The processing instruction sheet is printed only when a processing region displayed by a display unit and a content of processing to be performed thereon are determine to be the same as those instructed by a user.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus forrecognizing a processing instruction written on a medium, an imageprocessing method, and a program.

2. Description of the Related Art

Conventionally, there is a technique in which a computer analyzes animage on a sheet of paper read by a scanner and recognizes text writtenon the paper. For example, such a technique can be used to easilyextract money amounts and dates written in a plurality of forms and toenter the extracted data to processing such as adding the data. However,instructing the computer on where the various types of data are writtenin the sheets to automatically perform such processing is to beperformed.

Japanese Patent Application Laid-Open No. 2008-145611 discusses atechnique in which an answer sheet to be processed and a processinginstruction sheet are read by a series of reading processes. Theprocessing instruction sheet describes processing instructioninformation for instructing which columns are to be processed and theprocessing to be performed on the contents written in the columns in theanswer sheet form. The processing instruction information is thendetected and analyzed based on the read data of the processinginstruction sheet. As a result, write-in regions which are to beprocessed on the form and processing contents for each region can berecognized.

However, according to the above-described method, users are unable toconfirm, before analyzing a document to be processed, whether anapparatus has correctly recognized the regions to be processed and theprocessing contents of the regions described in the processinginstruction sheet.

SUMMARY OF THE INVENTION

According to an aspect of the present invention, an apparatus includes areading unit configured to read image data, a recognition unitconfigured to recognize a region designated by a handwritten portion andprocessing associated with a color of the handwritten portion in theimage data, and a display unit configured to display a preview bysuperimposing a recognized result and the region to be processed in theimage data and displaying a content of the recognized processing on thedisplayed preview.

Further features and aspects of the present invention will becomeapparent from the following detailed description of exemplaryembodiments with reference to the attached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate exemplary embodiments, features,and aspects of the invention and, together with the description, serveto explain the principles of the invention.

FIG. 1 is a block diagram illustrating a configuration of an imageprocessing apparatus according to an exemplary embodiment of the presentinvention.

FIGS. 2A, 2B, and 2C illustrate examples of documents to be processedaccording to an exemplary embodiment of the present invention.

FIG. 3 is a flowchart illustrating a process for generating a scanticket according to an exemplary embodiment of the present invention.

FIG. 4 is a flowchart illustrating a process for checking a documentusing a scan ticket according to an exemplary embodiment of the presentinvention.

FIG. 5 illustrates an enlarged portion of the processing instructionsheet.

FIG. 6 illustrates an example of a Quick Response (QR) code.

FIG. 7 is a flowchart illustrating a thumbnail image generation process.

FIG. 8 illustrates an example of a thumbnail image.

FIG. 9 illustrates an example of a scan ticket.

FIG. 10 is a flowchart illustrating an instruction color change process.

FIGS. 11A and 11B illustrate examples of instruction color change forms.

FIG. 12 is a flowchart illustrating an instruction color change process.

FIG. 13 illustrates an example of a thumbnail image.

FIG. 14 is a flowchart illustrating an instruction color change process.

FIG. 15 illustrates an example of a thumbnail image.

FIG. 16 illustrates an example of a scan ticket.

DESCRIPTION OF THE EMBODIMENTS

Various exemplary embodiments, features, and aspects of the inventionwill be described in detail below with reference to the drawings.

FIG. 1 is a block diagram illustrating a configuration of an imageprocessing apparatus according to a first exemplary embodiment of thepresent invention. Referring to FIG. 1, an image processing apparatus100 is a multifunction peripheral including various functions such as acopy function and a scanner function. The functions can also be realizedby a plurality of apparatuses in cooperation with each other.

A central processing unit (CPU) 11 controls the entire operations of theimage processing apparatus 100 by loading programs stored in a read-onlymemory (ROM) 19 onto a random access memory (RAM) 18. Further, the CPU11 communicates with each of the components in the image processingapparatus 100 via a bus 12. An operation unit 16 includes a plurality ofkeys used by the user to give instructions, and a display unit fordisplaying various types of information to be informed to the user. Ascanner 15 reads as a color image an image on a document that the usersets on a document positioning plate and stores the acquired electronicdata (image data) in a hard disk drive (HDD) 13 or the RAM 18. The HDD13 includes a hard disk and stores various input information. Thescanner 15 also includes a document feeding device, so that a pluralityof documents set on the document feeding device can be sequentially fedto the document positioning plate and be read by the scanner 15. Aprinter 14 prints on a recording sheet an image based on the input imagedata. A network interface (I/F) 17 connects the image forming apparatus100 to a network 20 and controls receiving and transmitting data to andfrom an external device on the network.

According to the present exemplary embodiment, the image data to be usedin performing processing to be described below is input via the scanner15. However, image data of a document transmitted from an externaldevice can be input via the network I/F 17 and be similarly processed.Further, a personal computer (PC) connected to a scanner and a printercan similarly perform processing. In such a case, the entire program ora portion of the program used in the present exemplary embodiment can besupplied to the PC via the network or by storing in a recording mediumsuch as a compact disc (CD)-ROM.

An example of a document to be used in the present exemplary embodimentwill be described below. FIG. 2A illustrates an example of a form of thedocument to be used in the present exemplary embodiment. Referring toFIG. 2A, the document is an invoice which has not been filled in (i.e.,before being added processing instruction information to be describedbelow) by the user. The invoice includes the following information: Nameof company to be billed, person in charge of the company to be billed,subject, total amount, item, quantity, unit, unit price, amount ofpayment, bank deposit information, sub-total, and tax includingconsumption tax. The invoice further includes a remarks column and aspace where an issuer places a seal. When officially issuing theinvoice, the user writes the information in the columns to indicate thesubject, items and quantity, units, unit prices, and amounts of payment,and places the seal.

According to the present exemplary embodiment, a process is performed tocheck whether the information is added to the columns designated by theuser and other columns are left blank in the invoice.

FIG. 2B illustrates an example in which the user has arbitrarily markedthe regions to be checked among the items included in the documentillustrated in FIG. 2A. The user marks the regions using color pens. Thedocument illustrated in FIG. 2B thus becomes the processing instructionsheet.

The processing instruction sheet is created by the user who is to checkthe created invoice. The user writes the processing instructioninformation to be described below in a sheet of a similar format as theinvoice to be checked. In other words, the processing instruction sheetis created by the user writing the processing instruction information inthe invoice illustrated in FIG. 2A. According to the present exemplaryembodiment, the user uses a color pen to designate the region to beprocessed by encircling the region with a rectangle. The region can bedesignated by drawing a line at the bottom of the region, alternatively.

The processing instruction information (i.e., additional information) tobe written in the invoice (i.e., document) will be described below.Referring to FIG. 2B, the user may mark a region 31 using a blue colorpen, a region 32 using a green color pen, and a region 33 using a redcolor pen. Colors other than the above can also be used, and the numberof colors is not limited to three and can be increased or decreasedaccording to the contents to be checked. Further, means other than thecolor pens can be used as long as color marks can be written on thedocument.

The user previously registers in the RAM 18 via the operation unit 16information about the colors of the processing instruction informationto be used and the processing contents associated with each other. Morespecifically, the user registers in the RAM 18 that the blue pen is usedto check whether the column is blank, the green pen to check whetherinformation is entered, and the red pen to check whether there is a sealor a signature. The CPU 11 then determines a color component (e.g., ahue) of each color registered in the RAM 18 and stores the determinedcontent therein. The color registration can also be performed by thescanner 15 reading the added information written in the form. Further,the image processing apparatus 100 can previously register the colorsinstead of the user. In a case where the process is to be performedaccording to the previously registered contents, the user adds theprocessing instruction information to the document according to theregistered colors and processing contents.

As described above, the processing instruction sheet is createdaccording to the registered color components of the processinginstruction information to be used and the corresponding processingcontents. The processing instruction information is thus extracted usingthe processing instruction sheet, and the processing contents arerecognized according to the extracted results. The image processingapparatus 100 then checks whether a specific region containsinformation, is blank, or is sealed.

FIG. 2C illustrates an example of a document to be checked according tothe present exemplary embodiment. Referring to FIG. 2C, the document tobe checked is of a similar format as the documents illustrated in FIGS.2A and 2B. According to the present exemplary embodiment, the addedprocessing instruction information as illustrated in FIG. 2B isextracted. The extracted result is then used to determine that thedocument is normal if a region 41 does not include any content, i.e.,the region 41 is blank, a region 42 includes content, and a region 43 issealed. Since FIG. 2C illustrates an example in which all conditions fordetermining that the document is normal are satisfied, the check resultof the document becomes normal. On the other hand, if even one regiondoes not satisfy the condition for determining that the document isnormal, the check result of the document becomes not normal. Thecontents and the regions to be checked are not limited to theabove-described contents and regions, and instructions can be issued tocheck other contents and regions.

A process for creating a scan ticket to check the contents described inthe document according to the processing instruction sheet illustratedin FIG. 2B will be described below. The image processing apparatus 100recognizes the contents of the instructions illustrated in FIG. 2B andconverts a method for checking the document to be checked such asillustrated in FIG. 2C (e.g., a QR code) into a form that can berecognized, i.e., the scan ticket. The scan ticket includes the contentsof the instructions recognized from the document illustrated in FIG. 2B,and position information of the regions to which such contents of theinstructions are to be applied. When the document is to be checked, thescanner 15 reads the scan ticket and the CPU 11 is then caused torecognize the processing contents.

The scan ticket creation process according to the present exemplaryembodiment will be described in detail below. FIG. 3 is a flowchartillustrating the scan ticket creation process according to the presentexemplary embodiment. Further, FIG. 5 illustrates an enlarged version ofthe processing instruction region 31, and FIG. 6 illustrates an exampleof the QR code in which the processing instruction information isencoded.

The flowchart of FIG. 3 illustrates the flow of a process performed bythe CPU 11 loading onto the RAM 18 and executing a program stored in theROM 19.

When the user instructs the image processing apparatus 100 via theoperation unit 16 to create the scan ticket, the process is started. Instep S501, the CPU 11 causes the operation unit 16 to display acombination of an instruction color of the processing instructioninformation (herein after referred to as instruction color) and theprocessing content. For example, the operation unit 16 displays “OK ifthere is a seal or signature in a region enclosed by red color”, “OK ifa region enclosed by blue color is blank”, or “OK if there is a writingin a region enclosed by green color”. In step S502, the CPU 11 furthercauses the operation unit 16 to display an inquiry to the user onwhether the combination of the instruction color and the processingcontent displayed in step S501 is correct.

If the user disagrees to the inquiry of step S502 via the operation unit16 (NO in step S502), the process proceeds to step S505. In step S505,the CPU 11 causes the operation unit 16 to display a message that thecombination of the instruction color and the processing content will bechanged. More specifically, the operation unit 16 may display an inquiryon which color to be changed to and display a new color instead of theinstruction color, or the user may designate an arbitrary color via theoperation unit 16. Further, the user may only be required to change thecombination of the color and the processing content instead of adding anew color. In such a case, since different processing contents cannot beinstructed with the same color, the CPU 11 performs control so that aprocessing content is paired with one color. Upon changing theinstruction color or the processing content or both the instructioncolor and the processing content in step S505, the CPU 11 causes theoperation unit 16 to display a combination of the instruction color andthe processing content as in step S501. As a result, the user canconfirm that the change process has been performed in step S505. If theuser then agrees to the inquiry of step S502 via the operation unit 16(YES in step S502), the CPU 11 determines and registers in the RAM 18the instruction color of the processing instruction information to beused and the corresponding processing content.

In the determination process of step S502, the user is caused tovisually confirm the content of the document (i.e., colors included inthe document). If the user then determines that the color component ofthe instruction color and a color component included in the document aresimilar, the color components are distinguished to become different fromeach other. An error in extracting the processing instructioninformation can thus be prevented.

Further, if it is determined in step S502 that the color component ofthe instruction color and the color component included in the documentare similar, the operation unit 16 is caused to display a message urgingthe user to set the document as will be described below. If it is thendetermined that the user has set the document, the document can becopied in monochrome. As a result, an error in extracting the processinginstruction information when the processing instruction information isadded using a chromatic color pen can be prevented.

When the user determines that the instruction color and the processingcontent are OK in step S502, the CPU 11 identifies the color componentto be used as the processing instruction information and stores thecolor component in the RAM 18. In step S503, the CPU 11 causes theoperation unit 16 to display an inquiry on whether the user only has thedocument to be checked at hand (i.e., illustrated in FIG. 2C). This isto confirm whether there is a document to be a template for creating theprocessing instruction sheet (i.e., illustrated in FIG. 2A or FIG. 2B).In other words, if the user only has the document to be checked, adocument for writing in the processing instruction information can becreated from the document to be checked as will be described below. Sucha template is not officially the document to be checked and is instead adocument in which the user can add in the processing instructioninformation. If the CPU 11 receives a response via the operation unit 16that the user only has the document to be checked (i.e., there is nodocument to be the template) (YES in step S503), the process proceeds tostep S504. In step S504, the CPU 11 causes the operation unit 16 todisplay a message urging the user to set the document to be checked onthe scanner 15. For example, the operation unit 16 displays a message“set one sheet of the document to be checked on the scanner and pressthe OK button” and an OK button. The CPU 11 recognizes that the documentis set when the user presses the OK button. The CPU 11 can alsoautomatically recognize that the document is set on the scanner 15 byusing the photo-interpreter below the document positioning plate or thedocument sensor of the document feeding device.

Upon determining that the user has pressed the OK button in step S504,the process proceeds to step S506. In step S506, the CPU 11 causes thescanner 15 to read the image in the document to be checked. The CPU 11further converts the image data input from the scanner 15 to monochromeimage data, outputs the monochrome image data to the printer 14, andcauses the printer 14 to output a monochrome copy on the recoding sheet.In step S506, the document is converted to monochrome and printed by theprinter 14. However, the present invention is not limited to the above,and a color of the image in the read document can be converted to colorsother than the instruction color and be printed by the printer 14. Forexample, the document is output by converting red characters in the readdocument to blue characters. Furthermore, colors to be converted can bepreviously registered in the RAM 18, and if a registered color exists inthe read document, the color can be converted.

In step S507, the CPU 11 causes the operation unit 16 to display amessage urging the user to write the processing instruction informationillustrated in FIG. 2B in the recording sheet output by the printer 14in step S506. On the other hand, if the CPU 11 receives a response thatthe user has the template document (NO in step S503), the processproceeds to step S508. In step S508, the CPU 11 causes the operationunit 16 to display an inquiry on whether the processing instructioninformation (as illustrated in FIG. 2B) is already written in thetemplate. If the user responds via the operation unit 16 that theinstruction information is not written in the template (NO in stepS508), the process proceeds to step S509. In step S509, the CPU 11causes the operation unit 16 to display a message urging the user to setthe template on the scanner 15. For example, the operation unit 16displays a message, i.e., “set the template on the scanner and press theOK button”, and the OK button. According to the present exemplaryembodiment, the CPU 11 recognizes that the document is set by the userpressing the OK button. However, the CPU 11 can automatically recognizethat the document is set on the scanner 15 by using thephoto-interpreter located below the document positioning plate or thedocument sensor of the document feeding device.

Upon the user pressing the OK button in step S509, the process proceedsto step S510. In step S510, the CPU 11 causes the scanner 15 to read theimage in the template document. In step S511, the CPU 11 analyzes theacquired image data and performs a recognition process for determiningwhether the image data includes a color having the same color componentas the instruction color. For example, if the CPU 11 is to analyze andrecognize whether the red color is included in the acquired image data,the CPU 11 extracts a red hue component. Various known methods foranalyzing and recognizing the hue component can be employed. Further,parameters other than the hue can be used, or a combination of otherparameters can be used.

In step S512, the CPU 11 determines whether there is a color among thecolors analyzed and recognized in step S511 which is the same as theinstruction color registered in the RAM 18. It is not necessary for thecolors to exactly match in the determination process, and the colors canbe determined to match if they are within an allowable range. Forexample, in a case where the red, green, and blue (RGB) values areexpressed in 256 gradations, the colors are determined to be the same asa result of comparison if the R, G, and B values of the recognized colorand the instruction color are within plus or minus 20 gradations. Therecognized color can be determined to be the same as the instructioncolor using a method other than the above-described method.

If the CPU 11 determines that the image of the template includes thesame color as the instruction color registered in the RAM 18 (YES instep S512), the process proceeds to step S513. In step S513, the CPU 11causes the operation unit 16 to display a message urging the user to setthe template on the scanner 15. For example, the operation unit 16displays a guiding message, i.e., “set the template on the scanner andpress the OK button”, and the OK button. According to the presentexemplary embodiment, the CPU 11 recognizes that the document is set bythe user pressing the OK button. However, the CPU 11 can automaticallyrecognize that the document is set on the scanner 15 by using thephoto-interpreter located below the document positioning plate or thedocument sensor of the document feeding device.

Upon the user pressing the OK button in step S513, the process proceedsto step S514. In step S514, the CPU 11 causes the scanner 15 to read theimage in the document to be checked. The CPU 11 then converts the imagedata input from the scanner 15 to monochrome image data, outputs theconverted image data to the printer 14, and causes the printer 14 tooutput a monochrome copy on the recording sheet. The present inventionis not limited to converting the document into a monochrome image andprint on the printer 14. Other methods as described above (i.e., as instep S506) can be employed.

In step S515, the CPU 11 causes the operation unit 16 to display amessage urging the user to write the processing instruction informationas illustrated in FIG. 2B on the recording sheet output by the printerin step S514.

On the other hand, if the CPU 11 determines that the template image doesnot include the same color as the instruction color registered in theRAM 18 (NO in step S512), the process proceeds to step S516. In stepS516, the CPU 11 causes the operation unit 16 to display a messageurging the user to write on the template document the processinginstruction information as illustrated in FIG. 2B.

In step S508, the CPU 11 causes the operation unit 16 to display amessage for confirming whether the processing instruction information isalready written in the template. If the CPU 11 receives via theoperation unit 16 a response that the instruction information is writtenin the template (YES in step S508), the process proceeds to step S517.In step S517, the CPU 11 causes the scanner 15 to read the image in thetemplate document in which the instructions are already written. Thescanner reads the document similarly as in the process for outputting amonochrome copy described above. More specifically, the CPU 11 causesthe operation unit 16 to display a message urging the user to set thedocument in which the instruction information is written. Upon the usersetting the document and pressing the OK button, the scanner 15 readsthe document. However, the image data read and acquired by the scanner15 is not converted to the monochrome image data and is stored in theRAM 18.

In step S518, the CPU 11 analyzes and recognizes the processinginstruction information in the image data input from the scanner 15. TheCPU 11 analyzes where the instruction color determined in step S502 isincluded in the document and identifies the position of the region to beprocessed for each color by recognizing the color of the analyzedportion. The position and the size of the region to be processed in thedocument can thus be determined by the identified position. For example,the position is identified using coordinates.

FIG. 5 illustrates a region corresponding to the processing instructionregion 31 illustrated in FIG. 2B which the user has written in using ablue color pen and is recognized in step S518 as the encircled region.Referring to FIG. 5, a starting point 701 in the upper left end, a width702, and a height 703 are extracted as check region information. When aposition in an X-Y coordinate system is expressed as (Y, X), thestarting point 701 is (1108, 963), the width 702 is 834, and the height703 is 128.

Similarly, in the case of the processing instruction region 32 writtenin by the checker with a green color pen, the values of the startingpoint, the width, and the height are (1240, 963), 740, and 100,respectively. Further, in the case of the processing instruction region33 written in by the checker with a red color pen, the values of thestarting point, the width, and the height are (2955, 2050), 200, and171, respectively. Furthermore, the instruction colors are determinedfrom the processing instruction regions 31, 32, and 33, and processingcodes are extracted. Since the processing instruction region 31 iswritten in using the blue color pen, the processing code is 1.Similarly, the processing code is 0 for the processing instructionregion 32 written in with the green color pen, and the processing codeis 2 for the processing instruction region 33 written in with the redcolor pen.

The CPU 11 then stores in the RAM 18 the identified positions associatedwith the processing contents determined in step S502.

In step S519, the CPU 11 causes the operation unit 16 to display theresult of the analysis and the recognition performed in step S518. Forexample, the operation unit 16 displays the coordinates and theprocessing contents of the regions corresponding to the identifiedprocessing instruction information. Further, the operation unit 16 maydisplay a thumbnail image of the read document, so that the user canrecognize the position of the processing instruction information withrespect to the image and the processing content. In step S520, the CPU11 causes the operation unit 16 to display a confirmation to the user onwhether the result of the analysis and the recognition displayed in stepS519 is correct. If the CPU 11 receives a response from the user via theoperation unit 16 that the result is incorrect (NO in step S520), theprocess proceeds to step S525. In step S525, the CPU 11 causes theoperation unit 16 to display a confirmation to the user on whether thetemplate image read by the scanner 15 in step S517 is to be output inmonochrome.

If the CPU 11 receives a response from the user via the operation unit16 to output the template image in monochrome (YES in step S525), theprocess proceeds to step S526. In step S526, the CPU 11 converts theimage of the document read by the scanner 15 in step S517 to monochromeimage data and causes the printer 14 to output the monochrome copy.

More specifically, if the processing instruction information cannot becorrectly extracted, the processing instruction sheet in which theprocessing instruction information is added is copied in monochrome. Theoutput is used so that the user can again add the processing instructioninformation. In step S526, the document is thus converted to monochromeand printed by the printer 14. However, the present invention is notlimited to such a method, and various methods as described above (i.e.,as in step S506) can be employed instead.

In step S527, the CPU 11 causes the operation unit 16 to display amessage urging the user to write the processing instruction informationin the recording sheet output by the printer 14 in step S526.

Returning to step S525, if the CPU 11 receives a response from the uservia the operation unit 16 not to output the template image in monochrome(NO in step S525), the process proceeds to step S528. In step S528, theCPU 11 causes the operation unit 16 to display a message confirmingwhether to create a new processing instruction sheet. If the userinstructs via the operation unit 16 to create a new processinginstruction sheet (YES in step S528), the process proceeds to step S529.In step S529, the CPU 11 causes the operation unit 16 to display amessage urging the user to set a newly created processing instructionsheet on the scanner 15. On the other hand, if the user instructs viathe operation unit 16 not to create a new processing instruction sheet(NO in step S528), the process ends.

If the user instructs the apparatus to read the document by setting thenewly created document and pressing the OK button after the messages aredisplayed in step S527 and step S529, the CPU 11 repeats the process ofstep S517 described above.

Returning to step S520, if the CPU 11 receives via the operation unit 16a response from the user that the result of the analysis is correct (YESin step S520), the CPU 11 stores in the RAM 18 the analyzed content asthe extraction result of the processing instruction information. In stepS521, the CPU 11 causes the operation unit 16 to display an inquiry onwhether to create the scan ticket. If the CPU 11 receives via theoperation unit 16 a response to create the scan ticket (YES in stepS521), the process proceeds to step S522. In step S522, the CPU 11encodes the analyzed content using, for example, a two-dimensional code(e.g., the QR code) to encode the analysis result displayed in stepS519. The region which is instructed to be processed and thecorresponding processing content are encoded.

For example, in the case of the processing instruction sheet illustratedin FIG. 2C, the processing instruction information of the processinginstruction regions 31, 32, and 33 are written as a character string.Such a character string includes the X-coordinate of the starting point,the Y-coordinate of the starting point, the width, the height, and theprocessing codes in such an order separated by commas and becomes asfollows: “963, 1108, 834, 128, 1, 963, 1240, 740, 100, 0, 2050, 2955,200, 171, 2”. FIG. 6 illustrates the above-described character stringencoded into the QR code.

According to the present exemplary embodiment, a two-dimensional code isdescribed as an example. However, the character string can be encodedusing other methods as long as the image processing apparatus 100 cananalyze and recognize such a code. In step S523, the CPU 11 causes theprinter 14 to output on the recording sheet and to print the encodedcharacter string created in step S522 as an image.

The CPU 11 can then use the printed scan ticket and check the documentto be checked. However, if the user determines that the result of theanalysis is correct in step S520, it indicates that the processinginstruction sheet which the CPU 11 caused the scanner 15 to read in stepS517 has been correctly recognized. It thus becomes unnecessary toperform the processes of step S521, step S522, and step S523, and theread processing instruction sheet may be used as the scan ticket. Insuch a case, the processing content is recognized from the processinginstruction sheet when performing the checking process.

Returning to step S521, if the CPU 11 receives via the operation unit 16a response not to create the scan ticket (NO in step S521), the CPU 11causes the operation unit 16 to display an identification (ID) foridentifying the analyzed content registered in step S520. Such a processis performed to identify and read from the RAM 18 the analyzed contentto be used in checking the check document. The user may designate an IDfrom the operation unit 16 instead of the CPU 11 proposing the ID. TheCPU 11 then stores in the RAM 18 the determined ID associated with theanalyzed content, and the process proceeds to step S524.

In step S524, the CPU 11 checks the document to be checked according tothe processing instruction information and the corresponding processingcontent recognized as described above.

According to the above-described process, the color component includedin the document is converted to become different from the colorcomponent of the instruction color. The document for the user to add theprocessing instruction information can then be printed. As a result, theprocessing instruction information added to the processing instructionsheet can be correctly recognized. In other words, errors in recognizingthe processing instruction information can be reduced. Further, sincethe operations for performing the above-described process can beappropriately displayed to the user (e.g., a message urging themonochrome copy to be output), errors generated by user operations canbe reduced. It is not necessary to perform all of the above-describedprocesses illustrated in the flowchart, and a portion of the processesmay be performed.

A procedure for checking the document according to the extractedprocessing instruction information using the scan ticket created asdescribed above (i.e., the process performed in step S524) will bedescribed below with reference to FIG. 4.

FIG. 4 is a flowchart illustrating a checking process performed by theimage processing apparatus according to the present exemplaryembodiment.

In step S2501, the CPU 11 causes the operation unit 16 to display aconfirmation on creating the scan ticket as will be described below.

In step S2502, the CPU 11 causes the operation unit 16 to display aninquiry on whether to create the scan ticket. If the CPU 11 receives viathe operation unit 16 a response to create the scan ticket (YES in stepS2502), the process proceeds to step S2503. On the other hand, if theCPU 11 receives via the operation unit 16 a response not to create thescan ticket from the checker who has already acquired the scan ticket(NO in step S2502), the process proceeds to step S2504 withoutperforming the scan ticket creation process.

In step S2503, the CPU 11 performs the scan ticket creation process inwhich the scan ticket is created from the above-described processinginstruction sheet. The process then proceeds to step S2504. The detailsof the scan ticket and the scan ticket creation process are as describedabove.

In step S2504, the CPU 11 causes the operation unit 16 to display thatthe document to be checked will be scanned after the scan ticket isfirst scanned and urges the user to set the documents on the scanner 15.

In step S2505, upon a sensor (not illustrated) detecting that thedocuments are set, the CPU 11 instructs the scanner 15 to read the scanticket and the document to be scanned, and the HDD 13 to store the imagedata. According to the present exemplary embodiment, there is only onedocument to be checked as illustrated in FIG. 2C for ease ofdescription. However, a plurality of documents can also be checked.

In step S2506, the CPU 11 reads the image data of the scan ticket storedin the HDD 13 and analyzes the scan ticket. A plurality of processinginstruction information is encoded in the QR code and printed in thescan ticket. Each of the processing instruction information includes thecheck region information indicating which region is to be checked, andthe processing code indicating the processing method for checking thecheck region. The CPU 11 detects the position of the QR code included inthe image data of the scan ticket, decodes the QR code, and acquires theplurality of processing instruction information. According to thepresent exemplary embodiment, the check region information of theprocessing instruction information indicates the coordinates on theimage data and includes the X-Y coordinates of a starting point of theprocessing region, and the width and height from the starting point.Further, the processing code which indicates the processing method forchecking the check region is a number corresponding to the processingmethod for the check region.

According to the present exemplary embodiment, three types of processingmethods correspond to the processing codes respectively, i.e., whetherthere is an entry in the check region (processing code: 0), whether thecheck region is blank (processing code: 1), and whether there is a redseal (processing code: 2). If the CPU 11 reads a plurality of documentsto be checked in step S2505, the processing instruction informationwritten in the scan ticket which is set as the first document is appliedto all subsequent documents to be checked. The processing instructioninformation is valid until the checking process is completed.

In step S2507, the CPU 11 sequentially reads the image data to bechecked stored in the HDD 13.

In step S2508, the CPU 11 selects one processing instruction informationfrom the plurality of processing instruction information. The CPU 11then sequentially acquires from the image data read in step S2507 thecheck region based on the starting point, the width, and the heightindicated by the check region information of the processing instructioninformation.

In step S2509, the CPU 11 confirms the processing code for the checkregion from the processing instruction information used in step S2508.If the processing code is 0 or 1, the process proceeds to step S2510. Ifthe processing code is 2, the process proceeds to step S2511.

In step S2510, the CPU 11 checks whether there is an entry in the checkregion. According to the present exemplary embodiment, the image datawithin the check region acquired in step S2508 is converted into a hue,lightness, and saturation (HLS) color space. Whether there is an entryin the check region can thus be determined based on a percentage ofpixels in the check region whose luminance L is lower than apredetermined brightness. According to the present exemplary embodiment,the image data of the check region acquired from the image data isdescribed in an RGB color space. Therefore, the percentage of the pixelsin the check region whose luminance L is less than 50% is acquired, andif the percentage of the pixels is 10% or greater, it is determined thatthere is an entry in the check region. The determination method is notlimited to the above as long as an entry can be determined. For example,an entry can be determined by a length of the dark pixels continuing inthe main scanning direction. Further, since the conversion from the RGBcolor space to the HLS color space is a known technique, detaileddescription is omitted.

In step S2511, the CPU 11 checks whether there is a red seal in theacquired check region. According to the present exemplary embodiment,the check region is converted into the HLS color space using a similarmethod as in step S2510. The CPU 11 then determines whether there is ared seal based on the percentage of pixels in the processing regionwhose saturation S and hue H are within a predetermined range. Morespecifically, according to the present exemplary embodiment, thepercentage of the pixels in the check region whose saturation S is 20%or greater and hue H is 30 degrees or lessor 330 degrees or greater isacquired. If the percentage of such pixels is greater than or equal to10%, the CPU 11 determines that there is a red seal. The determinationmethod is not limited to the above-described method as long as it can bedetermined whether there is the red seal. For example, the red seal canbe determined by the length of pixels continuing in the main scanningdirection whose saturation S and hue H are within a predetermined range.

In step S2512, the CPU 11 determines whether the percentage acquired instep S2510 or step S2511 is greater than or equal to a predeterminedvalue, and the processing code of the check region is 0 or 2. If thepercentage and the processing code match the above-described conditions(YES in step S2512), the process proceeds to step S2513. On the otherhand, if the percentage and the processing code do not match theabove-described conditions (NO in step S2512), the process proceeds tostep S2514.

In step S2514, the CPU 11 determines whether the percentage acquired instep S2510 or step S2511 is less than a predetermined value, and theprocessing code of the check region is 1. If the percentage and theprocessing code match the above-described conditions (YES in stepS2514), the process proceeds to step S2513. If the percentage and theprocessing code do not match the above-described conditions (NO in stepS2514), the process proceeds to step S2515.

In step S2513, the CPU 11 stores in the RAM 18, i.e., the work memory,that the result of checking the check region acquired in step S2508 isOK.

In step S2515, the CPU 11 stores in the RAM 18 that the result ofchecking the check region acquired in step S2508 is not OK (NG).

In step S2516, the CPU 11 determines whether all of the plurality ofprocessing instruction information are checked for the image data readin step S2507. If all of the plurality of processing instructioninformation are checked (YES in step S2516), the process proceeds tostep S2517. If the checking process is not completed (NO in step S2516),the process returns to step S2508.

In step S2517, the CPU 11 determines whether processing of all of theimage data of the document to be checked read in step S2505 has beencompleted. If the processing has been completed (YES in step S2517), theprocess proceeds to step S2518. If the processing has not been completed(NO in step S2517), the process returns to step S2507.

In step S2518, upon completing the process on all check regions for theimage data of all documents to be checked, the CPU 11 transmits thecheck results and an instruction to display the check results to theoperation unit 16. The checking process then ends. The check result tobe displayed on the operation unit 16 is the document to be checked readin step S2505, and if even one result of checking the check region is“NO”, the operation unit 16 notifies that the result of the checkingresult according to the present exemplary embodiment is “NO”. Thedisplay content of the check result and the method for displaying thecheck result are not limited to the above-described content and method,and more detailed information can be displayed. Further, the image datacan be generated from the check result and transmitted to the printer 14so that a report is output.

The thumbnail image generation process will be described in detailbelow.

FIG. 7 is a flowchart illustrating the thumbnail image generationprocess according to the present exemplary embodiment in step S519illustrated in FIG. 3. FIG. 8 illustrates an example of the thumbnailimage.

The process illustrated in the flowchart can be realized by the CPU 11loading the program stored in the ROM 19 onto the RAM 18 and executingthereon. In step S901, the CPU 11 sequentially deletes from the imagedata of the processing instruction sheet the rectangles written in bythe checker using the color pens to instruct processing. Morespecifically, the CPU 11 uses the processing instruction informationacquired in step S518 to identify the region in the image datacorresponding to the rectangle written in to instruct processing. TheCPU 11 then replaces the pixels of the instruction color included in theregion with a background color of the processing instruction sheet andthus deletes the rectangles for instructing processing from the imagedata.

In step S902, the CPU 11 uses the processing instruction informationacquired in step S518 to draw a rectangle configured by linear lines inthe region of the image data in which the rectangle written in toinstruct processing is deleted. The position where the rectangle is tobe drawn can be determined from the information about the startingpoint, the width, and the height included in the check regioninformation of the processing instruction information. According to thepresent exemplary embodiment, colors in a defined range are set to theinstruction colors, so that a color whose luminance L is 50%, saturationS is 100%, and hue H is at the center of the range of the instructioncolor is used to draw the rectangle. For example, in the processinginstruction sheet illustrated in FIG. 2B, the rectangle in theprocessing instruction region 31 written in by the checker is deleted. Arectangle 1001 as illustrated in FIG. 8 is then drawn using a blue colorpen as the instruction color according to the processing instructioninformation. Similarly, the rectangles written in by the checker as theprocessing instruction regions 32 and 33 are deleted, and a rectangle1002 is drawn using the green color pen and a rectangle 1003 is drawnusing the red color pen as instruction colors according to theprocessing instruction information.

In step S903, the CPU 11 determines whether the above-described processhas been performed for all processing instruction information acquiredin step S518 of the flowchart illustrated in FIG. 3. If the process iscompleted (YES in step S903), the process proceeds to step S904, and ifthe process is not completed (NO in step S903), the process returns tostep S901.

In step S904, the CPU 11 reduces the image data acquired by performingthe processes from step S901 to step S903 and generates a thumbnailimage 1000. More specifically, the CPU 11 generates the thumbnail imageto be displayed on the operation unit 16 by reducing the image data to asize for displaying on the operation unit 16 in step S519 of theflowchart illustrated in FIG. 3. Further, the CPU 11 reduces the imagedata in which the rectangles of processing instruction acquired by theprocesses performed from step S901 to step S903 are deleted to a sizefor generating the scan ticket in step S523. The CPU 11 thus generatesthe thumbnail image for generating the scan ticket, and the process thenends. According to the present exemplary embodiment, a bi-cubic methodis used to reduce the image data. However, the method is not limited tothe bi-cubic method, and a nearest neighbor method can also be used.

The scan ticket generated by the scan ticket generation process in stepS523 of the flowchart illustrated in FIG. 3 will be described in detailbelow with reference to FIG. 9. FIG. 9 illustrates an example of thescan ticket generated from the processing instruction sheet illustratedin FIG. 2B.

The scan ticket includes a QR code 800 illustrated in FIG. 6 in whichthe processing instruction information is encoded, the thumbnail image1000, and processing instruction information 1101, 1102, and 1103. Theprocessing instruction information 1101 indicates the check regioninformation (i.e., the coordinates of the starting point, the width, andthe height of the check region) acquired from the processing instructionregion 31 illustrated in FIG. 2B. The processing instruction information1102 indicates the check region information and the processing methodcorresponding to the processing code acquired from the processinginstruction region 32. The processing instruction information 1103indicates the check region information and the processing methodcorresponding to the processing code acquired from the processinginstruction region 33. The thumbnail image 1100 is generated from theprocessing instruction sheet illustrated in FIG. 2B acquired by thethumbnail generation process. As described above, the thumbnail image isthe image data in which the rectangles written in by the checker on theprocessing instruction sheet to instruct processing have been deletedand in which the rectangles are drawn using the instruction coloraccording to the processing instruction information. Such image data isreduced to be included in the scan ticket and thus becomes the thumbnailimage 1000.

The instruction color change process will be described in detail withreference to FIGS. 10 and 11.

FIG. 10 is a flowchart illustrating the instruction color change processaccording to the present exemplary embodiment. FIGS. 11A and 11Billustrate examples of an instruction color change form according to thepresent exemplary embodiment. The process illustrated in the flowchartcan be realized by the CPU 11 loading the program stored in the ROM 19onto the RAM 18 and executing thereon.

In step S1201, upon the user inputting an instruction to the operationunit 16 to perform the instruction color change process, the CPU 11transmits to the printer 14 the image data of the instruction colorchange form illustrated in FIG. 11A. The CPU 11 then informs the printer14 to print the image data. Further, the CPU 11 transmits to theoperation unit 16 an instruction to display a message to the user tofill in the instruction color change form with an arbitrary color pen.The instruction color change form contains a plurality of methods forprocessing the check regions that can be executed by the imageprocessing apparatus 1000, and instruction color designation regions forassociating the instruction color with the processing method. A blackcolor (luminance L: 0%) rectangle is drawn around each instruction colordesignation region. An instruction color designation region 1301illustrated in FIG. 11A corresponds to a process for checking whetherthere is an entry in the check region (i.e., processing code: 0). Asdescribed above, green (in which the hue H is between 90 degrees and 150degrees and the saturation S is greater than or equal to 20%) ispreviously associated with the processing method as the instructioncolor. An instruction color designation region 1302 corresponds to aprocess for checking whether the check region is blank (i.e., processingcode: 1). Blue (in which the hue H is between 210 degrees and 270degrees and the saturation S is greater than or equal to 20%) ispreviously associated with the processing method as the instructioncolor. An instruction color designation region 1303 corresponds to aprocess for checking whether there is a red seal in the check region(i.e., processing code: 2). Red (in which the hue H is between 30degrees and 330 degrees and the saturation S is greater than or equal to20%) is previously associated with the processing method as theinstruction color.

In step S1202, the CPU 11 transmits to the operation unit 16 aninstruction to display a message to the user to scan the instructioncolor change form which the user has filled in using an arbitrary colorpen.

In step S1203, the CPU 11 transmits instructions to the scanner 15(i.e., reading unit) to read the instruction color change form and tothe HDD 13 to store the image data.

In step S1204, the CPU 11 (including a recognition unit) acquires fromthe HDD 13 the read image data of the instruction color change form andacquires the color information from the image data of the instructioncolor designation region. FIG. 11B illustrates an example of theinstruction color change form which the user has filled in using thearbitrary color pen and which has been read by the scanner 15. Referringto FIG. 11B, the instruction color change regions in the instructioncolor change form include regions 1304, 1305, and 1306 drawn using thecolor pens. The color information can be converted to the HLS colorspace after detecting the pixels of a color other than the color of therectangle drawn around the instruction color designation region andacquiring an average value of the pixels in the RGB color space.According to the present exemplary embodiment, the background color isideally white, which is (255, 255, 255) in the RGB color space, andwhose luminance L is 100% and saturation is 0% in the HLS color space.Further, the color of the rectangle drawn around the instruction colordesignation region is black, which is (0, 0, 0) in the RGB color space,and whose luminance L and saturation are 0% in the HLS color space. Itis not necessary for the color of the rectangle drawn around theinstruction color designation region to be black.

In step S1205, the CPU 11 determines whether the color information hasbeen read from all of the plurality of the instruction color designationregions. If the color information has been read from all of theplurality of the instruction color designation regions (YES in stepS1205), the process proceeds to step S1206. If the color information hasnot been read from all of the plurality of the instruction colordesignation regions (NO in step S1205), the process returns to stepS1204.

In step S1206, the CPU 11 determines whether there is an invalid coloramong the read color information. The color is determined as invalid ifthe colors within the instruction color designation region are only thebackground color and the color of the rectangle drawn around theinstruction color designation region. Further, the color is determinedas invalid if the read color information is similar to theabove-described two colors. If the CPU 11 determines that there is atleast one invalid color among the read color information (YES in stepS1206), the process proceeds to step S1207. On the other hand, if thereis no invalid color (NO in step S1206), the process proceeds to stepS1208. According to the present exemplary embodiment, white (i.e., theluminance L is 100% and the saturation S is 0%) and black (i.e., theluminance L and the saturation S are 0%) are invalid instruction colors.The colors whose saturation is less than or equal to 10% and whoseluminance L is between 10% and 90% are thus determined to be colorswhich are similar to the above-described colors.

In step S1207, the CPU 11 transmits to the operation unit 16 (which mayinclude a display unit) an instruction to display a message to the userto select whether to repeat the instruction color change process. Uponreceiving an instruction to repeat the process from the operation unit16 (YES in step S1207), the CPU 11 replaces the invalid colorinformation with a preset instruction color. The process then proceedsto step S1208. More specifically, if the invalid color is read from theinstruction color designation region 1301, a process to be describedbelow is performed by setting as the read color information a value atthe center of the range of the hue H of the green color. Green is presetto the processing method corresponding to the instruction colordesignation region 1301.

In step S1208, the CPU 11 determines whether there are similar colorsamong the plurality of color information. If there are similar colors(YES in step S1208), the process proceeds to step S1209. If there are nosimilar colors (NO in step S1208), the process proceeds to step S1211.According to the present exemplary embodiment, if the hue H of each ofthe color information is within ±30 degrees, the CPU 11 determines thatthere are similar colors.

In step S1209, the CPU 11 transmits to the operation unit 16 aninstruction to display a message to the user to select whether to repeatthe instruction color change process. Upon receiving an instruction torepeat the process from the operation unit 16 (YES in step S1209), theprocess returns to step S1201. On the other hand, if the CPU 11determines that an instruction is received to not repeat the process (NOin step S1209), the process proceeds to step S1210.

In step S1210, the CPU 11 reduces the range of the hue H for determiningthe set instruction color to which the color information is determinedto be similar among the plurality of color information.

In step S1211, the CPU 11 enlarges the range of the plurality of colorinformation acquired from the instruction color designation region to apredetermined range. The CPU 11 then sets each of the plurality of colorinformation as the instruction color corresponding to the respectiveprocessing method and changes the instruction colors. According to thepresent exemplary embodiment, the instruction color is determined bysetting the hue H in the color information to a range of ±30 degrees andthe saturation S in the color information to be greater than or equal to20%. If the color information is of the instruction color whose range isreduced in step S1210, the range of the hue H is reduced from ±30degrees. The instruction color is then set so that similar colors do notoverlap.

For example, in the instruction color change form illustrated in FIG.11B, the region 1304 is drawn using an orange color pen (i.e., hue H is30 degrees, saturation S is 100%, and luminance L is 100%). Further, theregion 1305 is drawn using a yellow color pen (i.e., hue H is 60degrees, saturation S is 100%, and luminance L is 100%), and the region1306 is drawn using a purple color pen (i.e., hue H is 270 degrees,saturation S is 100%, and luminance L is 100%). The CPU 11 then readsthe color information of the regions 1304, 1305, and 1306 in step S1204,and in step S1208, the CPU 11 determines that orange and yellow aresimilar colors. If the user does not select to repeat the process instep S1209, the process proceeds to step S1210, and the CPU 11 reducesthe setting ranges of the hue of each color to ±15 degrees. In stepS1211, the CPU 11 changes the instruction colors to correspond to theabove-described colors. In other words, orange (i.e., hue H is greaterthan or equal to 15 degrees and less than 45 degrees, and saturation Sis greater than or equal to 20%) becomes the instruction colorassociated with the process for checking whether there is an entry inthe check region (processing code: 0). Further, yellow (i.e., hue H isgreater than or equal to 45 degrees and saturation S is greater than orequal to 20%) becomes the instruction color associated with the processfor checking whether the check region is blank (processing code: 1).Furthermore, purple (i.e., hue H is between 240 degrees and 300 degreesand saturation S is greater than or equal to 20%) becomes theinstruction color associated with the process for checking whether thereis a red seal in the check region (processing code: 2).

As described above, according to the first exemplary embodiment, theimage processing apparatus reads the processing instruction sheet inwhich the checker has written in the regions and acquires the processinginstruction information. At the same time, the image processingapparatus deletes the regions and uses the processing instructioninformation to visually supply to the user the recognized processinginstruction information by drawing the rectangles using the instructioncolors. As a result, when performing the checking process, the checkercan easily and accurately recognize whether the processing method is tobe applied to the region.

According to the present exemplary embodiment, the processinginstruction information is converted to the QR code in the scan ticketgeneration process. However, the method for encoding the processinginstruction information is not limited to the above, and, for example, acolor bar code can be used for encoding. In such a case, the processinginstruction information is acquired by performing a decoding processcorresponding to the encoding method instead of decoding the QR code instep S2506 illustrated in FIG. 4.

A second exemplary embodiment of the present invention is different fromthe first exemplary embodiment in a portion of the thumbnail imagegeneration process illustrated in FIG. 7. Description on the portionswhich are similar to the first exemplary embodiment is thus omitted, andonly the different portion will be described.

FIG. 12 is a flowchart illustrating the thumbnail image generationprocess according to the present exemplary embodiment. The thumbnailimage generation process according to the present exemplary embodimentwill be described below with reference to FIG. 12.

In step S901, the CPU 11 sequentially deletes from the image data of theprocessing instruction the rectangles written in by the checker usingthe color pens to instruct processing.

In step S902, the CPU 11 uses the processing instruction informationacquired in step S518 to draw (display) the rectangle configured bylinear lines in the region of the image data in which the rectanglewritten in by the checker has been deleted.

In step S1401, the CPU 11 uses the acquired processing instructioninformation to draw the processing method corresponding to theprocessing code in the region of the image data in which the linear-linerectangle is drawn with the instruction color. The CPU 11 draws theprocessing method as text using the instruction color. FIG. 13illustrates an example of the thumbnail image according to the presentexemplary embodiment. Referring to FIG. 13, the CPU 11 draws in theregion of the rectangle 1001 drawn in step S902 the processing method ofthe processing code corresponding to the region, i.e., “blank” in theform of text 1501. Further, the CPU 11 draws in the region of therectangle 1002 the processing method of the processing codecorresponding to the region, i.e., “entry” in the form of text 1502.Furthermore, the CPU 11 draws in the region of the rectangle 1003 theprocessing method of the processing code corresponding to the region,i.e., “seal” in the form of text 1503.

In step S903, the CPU 11 determines whether the above-described processhas been performed for all processing instruction information acquiredin step S518 of the flowchart illustrated in FIG. 3. If the process iscompleted (YES in step S903), the process proceeds to step S904, and ifthe process is not completed (NO in step S903), the process returns tostep S901.

In step S904, the CPU 11 reduces the image data acquired by performingthe processes from step S901 to step S903 and generates the thumbnailimage. The thumbnail image generation process thus ends.

The scan ticket created according to the present exemplary embodimentwill be described below.

The thumbnail image acquired by performing the thumbnail imagegeneration process illustrated in FIG. 12 is used in step S519 and stepS523 in the flowchart of the scan ticket creation process illustrated inFIG. 3. For example, in the present exemplary embodiment, a thumbnailimage 1500 illustrated in FIG. 13 is employed in a region of thethumbnail image 1100 illustrated in FIG. 9.

As described above, according to the second exemplary embodiment, inaddition to drawing the rectangle with the instruction color on thethumbnail image, the processing method to be applied to the region isexplicitly displayed by text. As a result, when performing the checkingprocess, the checker can easily and accurately recognize whether theprocessing method is to be applied to the region.

A third exemplary embodiment of the present invention is different fromthe first exemplary embodiment in a portion of the thumbnail imagegeneration process illustrated in FIG. 7. Description on the portionswhich are similar to the first exemplary embodiment is thus omitted, andonly the different portion will be described.

FIG. 14 is a flowchart illustrating the thumbnail image generationprocess according to the present exemplary embodiment. The thumbnailimage generation process according to the present exemplary embodimentwill be described below with reference to FIG. 14.

In step S901, the CPU 11 sequentially deletes from the image data of theprocessing instruction the rectangles written in by the checker usingthe color pens to instruct processing.

In step S902, the CPU 11 uses the processing instruction informationacquired in step S518 to draw (display) a rectangle configured of linearlines in the region of the image data in which the rectangle written inby the checker has been deleted.

In step S1601, the CPU 11 uses the acquired processing instructioninformation to draw a processing instruction information number in theregion of the image data in which the linear-line rectangle is drawnwith the instruction color. The CPU 11 draws the processing instructioninformation number using the instruction color. The processinginstruction information number is a number indicating the order in whichthe processing instruction information is acquired in step S518. FIG. 15illustrates an example of the thumbnail image according to the presentexemplary embodiment. Referring to FIG. 15, the CPU 11 draws aprocessing instruction information number 1701 corresponding to theregion of the rectangle 1001 drawn in step S902 as character “(1)”.Further, the CPU 11 draws in the region of the rectangle 1002 theprocessing instruction information number 1702 corresponding to theregion as character “(2)”. Furthermore, the CPU 11 draws in the regionof the rectangle 1003 the processing instruction information number 1703corresponding to the region as character “(3)”.

In step S903, the CPU 11 determines whether the above-described processhas been performed for all processing instruction information acquiredin step S518. If the process is completed (YES in step S903), theprocess proceeds to step S904, and if the process is not completed (NOin step S903), the process returns to step S901.

In step S904, the CPU 11 reduces the image data acquired by performingthe processes from step S901 to step S903 and generates a thumbnailimage. The thumbnail image generation process thus ends.

The scan ticket created according to the present exemplary embodimentwill be described below with reference to FIG. 16.

FIG. 16 illustrates an example of the scan ticket according to thepresent exemplary embodiment.

Referring to FIG. 16, the scan ticket includes the QR code 800illustrated in FIG. 6 in which the processing instruction information isencoded, a thumbnail image 1700, and processing instruction information1801, 1802, and 1803. The processing instruction information 1801includes the check region information (i.e., the coordinates of thestarting point, the width, and the height of the check region) acquiredfrom the processing instruction region 31 illustrated in FIG. 2B. Theprocessing instruction information 1801 also includes the processingmethod corresponding to the processing code and a processing instructioninformation number 1804 as character “(1)”. Similarly, the processinginstruction information 1802 includes a processing instructioninformation number 1805 as character “(2)”, and the processinginstruction information 1803 includes a processing instructioninformation number 1806 as character “(3)”. The thumbnail image 1700 isgenerated by the thumbnail generation process illustrated in FIG. 14. Asdescribed above, the thumbnail image 1700 is the image data in which therectangles written in by the checker on the processing instruction sheetto instruct processing have deleted and on which rectangles and theprocessing instruction information numbers have been drawn using theinstruction colors according to the processing instruction information.Such image data is reduced to be included in the scan ticket and thusbecomes the thumbnail image 1700.

As described above, according to the third exemplary embodiment, therectangles are drawn using the instruction colors on the thumbnailimage. Further, the rectangles drawn using the instruction colors areassociated with detailed processing instruction information of theregions in the scan ticket. As a result, the checker can easily andaccurately recognize, when performing the checking process, whether theprocessing method is to be applied to the region.

The present invention can also be achieved by providing a storage mediumwhich stores software (program code) for implementing functions of theabove-described exemplary embodiments. The program code stored in thestorage medium can be read and executed by a computer. In this case, thesoftware (program code) itself realizes the functions of theabove-described exemplary embodiments. The software (program code)itself and the storage medium that stores the software (program code)constitute the present invention.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all modifications, equivalent structures, and functions.

This application claims priority from Japanese Patent Application No.2009-056318 filed Mar. 10, 2009, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image processing apparatus comprising at least one processor which executes a program stored in a memory, wherein the at least one processor functions as: an input unit configured to input image data by reading a processing instruction sheet in which processing instruction information is written for instructing to perform processing on a document to be processed; a recognition unit configured to analyze a region designated by a handwritten portion and a color used for encircling the region by the handwritten portion and recognize a content of processing to be performed inside the analyzed region encircled with the analyzed color according to a pre-registered combination of a color and a content of processing to be processed inside an encircled region; a display control unit configured to display the analyzed region to be processed in the image data and the recognized content of the processing to be performed inside a region of the document corresponding to the displayed region; and a print control unit configured not to print an instruction sheet for instructing to perform the recognized processing on the analyzed region in the document to be processed which is input in the image processing apparatus, in a case where it is determined by a user that a recognition result of the analyzed region displayed by the display control unit is a same as the region designated by the handwritten portion and the recognized processing displayed by the display control unit is a same as processing corresponding to a color used for encircling the region, and to print the instruction sheet, in a case where it is determined by a user that a recognition result of the analyzed region displayed by the display control unit is different from the region designated by the handwritten portion and/or the recognized processing displayed by the display control unit is different from processing corresponding to a color used for encircling the region.
 2. The image processing apparatus according to claim 1, wherein the recognition unit is configured to delete, when the display control unit displays the recognized result, the handwritten portion in the image data.
 3. The image processing apparatus according to claim 1, wherein the display control unit is configured to display the recognized region using the color used in the handwritten portion.
 4. A method for controlling an image processing apparatus comprising: inputting image data by reading a processing instruction sheet in which processing instruction information is written for instruction to perform processing on a document to be processed; analyzing a region designated by a handwritten portion and a color used for encircling the region by the handwritten portion and recognizing a content of processing to be performed inside the analyzed region encircled with the analyzed color; controlling to display the analyzed region to be processed in the image data and the recognized content of the processing to be performed inside a region of the document corresponding to the displayed region; and controlling not to print an instruction sheet for instructing to perform the recognized processing on the analyzed region in the document to be processed which is input in the image processing apparatus, in a case where it is determined by a user that a recognition result of the analyzed region displayed by the display control unit is a same as the region designated by the handwritten portion and the recognized content of the processing, which are displayed a same as processing corresponding to a color used for encircling the region, and to print the instruction sheet, in a case where it is determined by a user that a recognition result of the analyzed region displayed by the display control unit is different from the region designated by the handwritten portion and/or the recognized processing displayed by the display control unit is different from processing corresponding to a color used for encircling the region.
 5. The method according to claim 4, further comprising deleting, when displaying the recognized result, the handwritten portion in the image data.
 6. The method according to claim 4, further comprising displaying the recognized region using the color used in the handwritten portion.
 7. A non-transitory computer-readable medium storing a program for causing a computer to execute the method according to claim
 4. 8. The non-transitory computer-readable medium according to claim 7, further comprising displaying the recognized region using the color used in the handwritten portion. 